The Relation of Speech and Gestures: Temporal Synchrony Follows Semantic Synchrony

The close relationship of speech and gestures becomes conspicuously obvious in the temporal coordination of both modalities. In this paper we investigate in how far temporal synchrony is affected by the semantic relationship of gestures and their lexical affiliates. The results showed that when both modalities redundantly express the same information, the gesture’s onset is closer to that of the accompanying lexical affiliate than when gestures convey complementary information: the closer speech and gestures are related semantically, the closer is their temporal relation. This novel finding is discussed with respect to implications for the production process of speech and gestures.